Repository navigation
Core SQL test harness: no runner retries, Raw header without escapes, heredocs in TopLevelWords - #623
Merged
Merged
Conversation
- Turn off the own retries of tests/clickhouse-test (--dont_retry_failures). Without it, the runner runs a failed test again when its output has one of its MESSAGES_TO_RETRY, and reports a pass as OK, so a flaky test can enter the allowlist. RETRY_FAILED is now the only retry. - Write the header of the Raw TSV formats (TSVRawWithNames, TabSeparatedRawWithNamesAndTypes, ...) without escapes, as the server does (TabSeparatedRowOutputFormat::writeLine). Live test against the server, with the escaped formats as a contrast. - Skip heredocs in TopLevelWords, as the statement splitter does, so that the words in a heredoc are not top-level words. Unit test. No test result of the allowlist changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: the run script gives the runner --dont_retry_failures, and a Raw format writes the header without escapes. - A splitter test for the heredoc fix of TopLevelWords: a FROM in a heredoc does not make `format y` the start of INSERT data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI review gate — AI review passedClean review from: Cursor Bugbot. |
polyglotAI-bot
marked this pull request as ready for review
November 19, 2026 13:58
There was a problem hiding this comment.
Auto-approved by the AI review gate on f353f05. Details in the approver's comment on this pull request.
AI review gate — auto-approved
History (latest 5 heads, newest first)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Three fixes for the core SQL test harness (
tests/clickhouse-test-runner/, merged in #621). A review of the same harness in the C# client (ClickHouse/clickhouse-cs#747) found them; the C++ shim has the same three problems. The change is in the harness only: the library (clickhouse/) does not change.tests/clickhouse-testruns a failed test again when its output has one of itsMESSAGES_TO_RETRY(for example "No such file or directory", "Connection reset by peer"), and reports a pass on the retry asOK(check_if_need_retry). So a flaky test can enter the allowlist, and no warning shows it.scripts/run-upstream-tests.shnow gives--dont_retry_failuresto the plain run and to theRETRY_FAILEDloop, soRETRY_FAILEDis the only retry. (The runner defines the flag after aparse_known_args()call, so--helpdoes not show it; it is accepted.)TSVRawWithNames,TSVRawWithNamesAndTypesand their aliases, the server writes the names and types lines without escapes, as it writes the rows (TabSeparatedRowOutputFormat::writeLine). The shim escaped the header always, so a column name with a tab or a backslash gavea\tbwhere the server gives a real tab.Runner::WriteBlocknow uses the Raw flag of the format for the header too.TopLevelWordsskips heredocs. The statement splitter and the FORMAT-clause finder skip$$...$$and$tag$...$tag$, butTopLevelWords(used to find the inline data ofINSERT ... SELECT ... FORMAT) skipped only quotes and comments: inINSERT INTO t SELECT $$x FROM y$$ ..., theFROMin the heredoc was a top-level word. It now skips a heredoc withHeredocEnd, as the splitter does.Tests
Live.HeaderOfFormatMatchesServer(live, shard 1 of CI): column names with a tab, a backslash, a quote, a newline and non-ASCII text, and an Enum type name with quotes and a backslash, inTSVRawWithNames,TabSeparatedRawWithNamesAndTypesandTSVRawWithNamesAndTypes. The expected text is the server's own output of the same query. Contrast:TSVWithNamesandTSVWithNamesAndTypes(escaped header). Without the fix, the three Raw queries fail and the two contrast queries pass.SqlText.TopLevelWordsSkipsLiteralsAndComments(unit): words in quotes, a$$heredoc, a$tag$heredoc and comments are skipped; a$that does not start a heredoc ($a$b$) does not skip the text after it.SplitStatements.Caseshas a new case for the effect on the splitter:INSERT INTO t SELECT $$ FROM $$ AS x, format y FROM s;is a statement of its own, notformat yfollowed by INSERT data. Without the fix, the two heredoc cases and the splitter case fail.-Werror);clang-format --dry-run --Werrorandshellcheck -xclean; the upstream tests that use a Raw format with names or a heredoc (00397_tsv_format_synonym,01948_heredoc,01957_heredoc_more,03580_heredoc_ambiguity, and 3 others) pass with--dont_retry_failures.allowlist.txtdoes not change.Reference: the C# fixes, ClickHouse/clickhouse-cs#747 (commit
bb8ef3af).🤖 Generated with Claude Code
Note
Low Risk
Changes are confined to the test harness and SQL text helpers; no library or production client paths, with regression tests added.
Overview
The clickhouse-test-runner harness gets three behavioral fixes aligned with upstream
clickhouse-clientand CI expectations.Upstream test runner retries are off by default:
run-upstream-tests.shpasses--dont_retry_failurestotests/clickhouse-test, so message-based automatic retries no longer turn flaky failures into[ OK ]. OptionalRETRY_FAILEDremains the only retry path; CI is documented as failing the job on the first failure.Raw TabSeparated name/type headers are written without escapes when the output format is Raw (
format.raw), matching the server forTSVRawWithNamesand similar formats instead of always usingWriteEscapedStringon the header.TopLevelWordsnow skips heredocs ($$...$$and$tag$...$tag$) viaHeredocEnd, so words inside heredocs are not treated as top-level SQL (fixing statement split /FORMATdetection for heredoc-heavyINSERTqueries).README and new live/unit tests (
HeaderOfFormatMatchesServer,TopLevelWordsSkipsLiteralsAndComments, split case with heredoc) cover these behaviors.Reviewed by Cursor Bugbot for commit f353f05. Bugbot is set up for automated code reviews on this repo. Configure here.